An Incremental Prefix Filtering Approach for the All Pairs Similarity Search Problem.

VSEncoding efficient coding and fast decoding of integer lists via dynamic programming.

K%2DModel A New Computational Model for Stream Processors.



Refreshing Models to Provide Timely Query Recommendations.
In this work we propose a comparative study of the effects of a continuous model update on the effectiveness of wellknown query recommendation algorithms. In their original formulation, these algorithms use static (i.e. pre-computed) models to generate recommendations. We extend these algorithms to generate suggestions using: a static model (no updates), a model updated periodically, and a model continuously updating (i.e. each time a query is submitted). We assess the results by previously proposed evaluation metrics and we show that the use of periodical and continuous updates of the model used for recommending queries provides better recommendations. 1.
Incremental Algorithms for Effective and Efficient Query Recommendation.

On Tag Spell Checking.
Word segmentation is a problem in several Asian languages that have no explicit word boundary delimiter, e.g. Chinese, Japanese, Korean and Thai. We propose to use featurebased approaches for Thai word segmentation.
Tuning the capacity of search engines Load%2Ddriven routing and incremental caching to reduce and balance the load.

Mining Query Logs Turning Search Usage Data into Knowledge.

Aging effects on query flow graphs for query suggestion.
World Wide Web content continuously grows in size and importance. Furthermore, users ask Web search engines to satisfy increasingly disparate information needs. New techniques and tools are constantly developed aimed at assisting users in the interaction with the Web search engine. Query recommender systems suggesting interesting queries to users are an example of such tools. Most query recommendation techniques are based on the knowledge of the behaviors of past users of the search engine recorded in query logs. A recent query-log mining approach for query recommendation is based on Query Flow Graphs (QFG). In this paper we propose an evaluation of the effects of time on this query recommendation model. As users interests change over time, the knowledge extracted from query logs may suffer an aging effect as new interesting topics appear. In order to validate experimentally this hypothesis, we build different query flow graphs from the queries belonging to a large query log of a real-world search engine. Each query flow graph is built on distinct query log segments. Then, we generate recommendations on different sets of queries. Results are assessed both by means of human judgments and by using an automatic evaluator showing that the models inexorably age.
Mining Query Logs.
We propose a new method to rank a special category of time-sensitive queries that are year qualified. The method adjusts the retrieval scores of a base ranking function according to time-stamps of web documents so that the freshest documents are ranked higher. Our method, which is based on feedback control theory, uses ranking errors to adjust the search engine behavior. For this purpose, we use a simple but effective method to extract year qualified queries by mining query logs and a time-stamp recognition method that considers titles and urls of web documents. Our method was tested on a commercial search engine. The experiments show that our approach can significantly improve relevance ranking for year qualified queries even if all the existing methods for comparison failed. 1
Challenges in designing an interest%2Dbased distributed aggregation of users in P2P systems.

Search shortcuts a new approach to the recommendation of queries.

Entry Pairing in Inverted File.

Search shortcuts driving users towards their goals.

Web Search Result Caching and Prefetching.

Special Section Scalable information systems.

Efficiency Issues in Information Retrieval Workshop.

Collaborative ranking of grid%2Denabled workflow service providers.

%28Query%29 History Teaches Everything%2C Including the Future.

SPRANKER A Discovery Tool to Rank Service Providers Using Quality of Experience.

ECIR 2008 Workshop on Efficiency Issues on Information Retrieval.

Design trade%2Doffs for search engine caching.

Proceedings of the 2nf International Conference on Scalable Information Systems

An Open Architecture for QoS Information in Business Grids.

Biological Experiments on the Grid A Novel Workflow Management Platform.

An Open Architecture for QoS Information in Business Grids.

Sorting Out the Document Identifier Assignment Problem.
Abstract. The compression of Inverted File indexes in Web Search Engines has received a lot of attention in these last years. Compressing the index not only reduces space occupancy but also improves the overall retrieval performance since it allows a better exploitation of the memory hierarchy. In this paper we are going to empirically show that in the case of collections of Web Documents we can enhance the performance of compression algorithms by simply assigning identifiers to documents according to the lexicographical ordering of the URLs. We will validate this assumption by comparing several assignment techniques and several compression algorithms on a quite large document collection composed by about six million documents. The results are very encouraging since we can improve the compression ratio up to 40 % using an algorithm that takes about ninety seconds to finish using only 100 MB of main memory. 1
Challenges on Distributed Web Retrieval.
In the ocean of Web data, Web search engines are the primary way to access content. As the data is on the order of petabytes, current search engines are very large centralized systems based on replicated clusters. Web data, however, is always evolving. The number of Web sites continues to grow rapidly and there are currently more than 20 billion indexed pages. In the near future, centralized systems are likely to become ineffective against such a load, thus suggesting the need of fully distributed search engines. Such engines need to achieve the following goals: high quality answers, fast response time, high query throughput, and scalability. In this paper we survey and organize recent research results, outlining the main challenges of designing a distributed Web retrieval system.
Load%2Dbalancing and caching for collection selection architectures.
Abstract — To address the rapid growth of the Internet, modern Web search engines have to adopt distributed organizations, where the collection of indexed documents is partitioned among several servers, and query answering is performed as a parallel and distributed task. Collection selection can be a way to reduce the overall computing load, by finding a trade-off between the quality of results retrieved and the cost of solving queries. In this paper, we analyze the relationship between the collection selection strategy, the effect on load balancing and on the caching subsystem, by exploring the design-space of a distributed search engine based on collection selection. In particular, we propose a strategy to perform collection selection in a load-driven way, and a novel caching policy able to incrementally refine the effectiveness of the results returned for each subsequent cache hit. The combination of load-driven collection selection and incremental caching strategies allows our system to retrieve two thirds of the top-ranked results returned by a baseline centralized index, with only one fifth of the computing workload. I.
Mining query logs to optimize index partitioning in parallel web search engines.

The impact of caching on search engines.

Know your neighbors web spam detection using the web topology.
Web spam can significantly deteriorate the quality of search engine results. Thus there is a large incentive for commercial search engines to detect spam pages efficiently and accurately. In this paper we present a spam detection system that uses the topology of the Web graph by exploiting the link dependencies among the Web pages, and the content of the pages themselves. We find that linked hosts tend to belong to the same class: either both are spam or both are non-spam. We demonstrate three methods of incorporating the Web graph topology into the predictions obtained by our base classifier: (i) clustering the host graph, and assigning the label of all hosts in the cluster by majority vote, (ii) propagating the predicted labels to neighboring hosts, and (iii) using the predicted labels of neighboring hosts as new features and retraining the classifier. The result is an accurate system for detecting Web spam that can be applied in practice to large-scale Web data.
Dynamic personalization of web sites without user intervention.

Special issue on heterogeneous and distributed IR.

Workshop on large%2Dscale distributed systems for information retrieval.
DHT systems are structured overlay networks capable of using P2P resources as a scalable platform for very large data storage applications. However, their efficiency expects a level of uniformity in the association of data to index keys that is often not present in inverted indexes. Index data tends to follow nonuniform distributions, often power law distributions, creating intense local storage hotspots and network bottlenecks on specific hosts. Current techniques like caching cannot, alone, cope with this issue. We propose a new distributed data structure based on a decentralized balanced tree to balance storage data and network load more uniformly across all hosts. The approach is stackable with standard DHTs and ensures that the DHT storage subsystem receives an uniform load by assigning fixed sized, or low variance, blocks. 1.
The query%2Dvector document model.

Query%2Ddriven document partitioning and collection selection.
Abstract — We present a novel strategy to partition a document collection onto several servers and to perform effective collection selection. The method is based on the analysis of query logs. We proposed a novel document representation called query-vectors model. Each document is represented as a list recording the queries for which the document itself is a match, along with their ranks. To both partition the collection and build the collection selection function, we co-cluster queries and documents. The document clusters are then assigned to the underlying IR servers, while the query clusters represent queries that return similar results, and are used for collection selection. We show that this document partition strategy greatly boosts the performance of standard collection selection algorithms, including CORI, w.r.t. a round-robin assignment. Secondly, we show that performing collection selection by matching the query to the existing query clusters and successively choosing only one server, we reach an average precision-at-5 up to 1.74 and we constantly improve CORI precision of a factor between 11 % and 15%. As a side result we show a way to select rarely asked-for documents. Separating these documents from the rest of the collection allows the indexer to produce a more compact index containing only relevant documents that are likely to be requested in the future. In our tests, around 52 % of the documents (3,128,366) are not returned among the first 100 top-ranked results of any query. I.
The social network of Java classes.

A privacy preserving web recommender system.



Toward a search architecture for software components.
The Grid and its related technologies enable large-scale sharing of resources of various types. We envision that in the near future applications will be completely built in a bottom-up fashion using software components deployed on various locations and interconnected to form a workflow graph. In this paper, we make some proposals on the design of a component search service, enabling users to locate the components they need to deploy an application. Copyright c ○ 2005 John Wiley &amp; Sons, Ltd. KEY WORDS: Grid computing; workflows; component rank; component search; PageRank 1.
A Grid Information Service Based on Peer%2Dto%2DPeer.
Abstract. Information Services are fundamental blocks of the Grid infrastructure. They are responsible for collecting and distributing information about resource availability and status to users: the quality of these data may have a strong impact on scheduling algorithms and overall performance. Many popular information services have a centralized structure. This clearly introduces problems related to information updating and fault tolerance. Also, in very large configurations, scalability may be an issue. In this work, we present a Grid Information Service based on the peerto-peer technology. Our system offers a fast propagation of information and has high scalability and reliability. We implemented our system complying to the OGSA standard using the Globus Toolkit 3. Our system can run on Linux and Windows systems, with different network configurations, so to trade off between redundancy (reliability) and cost. Keywords: Grid information service, Grid middleware, Peer-to-peer. 1
Component Metadata Management and Publication for the Grid.
There is growing attention to component-oriented software design of Grid applications. Within this framework, applications are built by assembling together independently developed software components. Two main approaches are commonly used to manage, develop and publish software components: one is based on an Interface Description Language (IDL); the other is typical, for instance, of Java and is based on introspection and design conventions. In this paper, we compare them and we propose a third approach that merges the flexibility and fast learning curve of the latter, with the rigor of the former. Our proposal is meant to help the transition towards more modern tools, which is required to develop versatile Grid applications. 1.
SIGIR workshop report the SIGIR heterogeneous and distributed information retrieval workshop.

An evaluation of component%2Dbased software design approaches.

A Highly Scalable Parallel Caching System for Web Search Engine Results.

WebDocs a real%2Dlife huge transactional dataset.

On%2Dline Generation of Suggestions for Web Users.
One important class of Data Mining applications is the so-called &#034;Web Mining&#034; that analyzes and extracts important and non-trivial knowledge from Web related data. Typical applications of Web Mining are represented by the personalization or recommender systems. These systems are aimed to extract knowledge from the analysis of historical information of a web server in order to improve the web site expressiveness in terms of readability and content availability. Typically, these systems are made up of two components. One, that is usually executed off-line with respect to the Web server normal operations, analyzes the server access logs in order to find a suitable categorization of users, the other, that is usually executed on-line with respect to the Web server normal operations, classifies the active requests according to the previous off-line analysis. In this paper we propose SUGGEST 2.0 a recommender system that, differently from those proposed so far, does not make use of any off-line component. Moreover, in the last part of the paper, we analyze the quality of the suggestions generated and the performance of our solution. To this purpose we also introduce a novel quality metric that tries to estimate the effectiveness of a recommender system as the capacity to anticipate user requests that could be issued farther in the future.
WINGS A Parallel Indexer for Web Contents.

An Effective Recommender System for Highly Dynamic and Large Web Sites.

Assigning document identifiers to enhance compressibility of Web Search Engines indexes.
Granting efficient accesses to the index is a key issue for the performances of Web Search Engines (WSE). In order to enhance memory utilization and favor fast query resolution, WSEs  use Inverted File (IF) indexes where the posting lists are stored as sequences of d gaps (i.e. differences among successive document identifiers) compressed using variable length encoding methods. This paper describes the use of a lightweight clustering algorithm aimed at assigning the identifiers to documents in a way that minimizes the average values of d gaps. The simulations performed on a real dataset, i.e. the Google contest collection, show that our approach allows to obtain an IF index which is, depending on the d gap encoding chosen, up to 23% smaller than the one built over randomly assigned document identifiers. Moreover, we will show, both analytically and empirically, that the complexity of our algorithm is linear in space and time.
An Online Recommender System.
Collaborative filtering techniques have been successfully employed in recommender systems in order to help users deal with information overload by making high quality personalized recommendations. However, such systems have been shown to be vulnerable to attacks in which malicious users with carefully chosen profiles are inserted into the system in order to push the predictions of some targeted items. In this paper we propose several metrics for analyzing rating patterns of malicious users and evaluate their potential for detecting such shilling attacks. Building upon these results, we propose and evaluate an algorithm for protecting recommender systems against shilling attacks. The algorithm can be employed for monitoring user ratings and removing shilling attacker profiles from the process of computing recommendations, thus maintaining the high quality of the recommendations.
Assigning Document Identifiers to Enhance Compressibility of Fulltext Indices.
F. Silvestri    , R. Perego    , S. Orlando    Istituto ISTI, Consiglio Nazionale delle Ricerche (CNR), Pisa, Italy    Dipartimento di Informatica, Universit a di Venezia, Mestre, Italy  Abstract. Index compression has been a major issue in the field of Information  Retrieval Systems. In particular, due to the impressive figures involved with Web  Search Engines (WSEs) the compression of the index is not an option anymore  but it has become a must. The most important index compression methods are  designed to work for Inverted File (IF) indexes. These methods are based on the  assumption that the posting lists are stored as sequences of d gaps (i.e. differences  among successive document identifiers). The compression is thus carried out by  using variable length encoding methods which represents smaller number using  a smaller number of bits. In this paper, instead of focusing on finding a novel  encoding method, we propose an algorithm which allows the assignment of identifiers  to documents in a way that minimizes the average values of d gaps. The  simulations performed on a real dataset, i.e. the Google contest collection, show  that our approach allows to obtain an IF index which is, depending on the d gap  encoding chosen, up to 23% smaller than the one built over randomly assigned  document identifiers. Moreover, we will show, both analytically and empirically,  that the complexity of our algorithm is linear in space and time.
Assigning identifiers to documents to enhance the clustering property of fulltext indexes.
Web Search Engines provide a large-scale text document retrieval service by processing huge Inverted File indexes. Inverted File indexes allow fast query resolution and good memory utilization since their d-gaps representation can be effectively and efficiently compressed by using variable length encoding methods. This paper proposes and evaluates some algorithms aimed to find an assignment of the document identifiers which minimizes the average values of d-gaps, thus enhancing the effectiveness of traditional compression methods. We ran several tests over the Google contest collection in order to validate the techniques proposed. The experiments demonstrated the scalability and effectiveness of our algorithms. Using the proposed algorithms, we were able to sensibly improve (up to 20.81%) the compression ratios of several encoding schemes.
An Online Recommender System for Large Web Sites.
In this paper we propose a WUM recommender system, called SUGGEST 3.0, that dynamically generates links to pages that have not yet been visited by a user and might be of his potential interest. Differently from the recommender systems proposed so far, SUGGEST 3.0 does not make use of any off-line component, and is able to manage Web sites made up of pages dynamically generated. To this purpose SUGGEST 3.0 incrementally builds and maintains historical information by means of an incremental graph partitioning algorithm, requiring no off-line component. The main innovation proposed here is a novel strategy that can be used to manage large Web sites. Experiments, conducted in order to evaluate SUGGEST 3.0 performance, demonstrated that our system is able to anticipate users&#039; requests that will be made farther in the future, introducing a limited overhead on the Web server activity .
A Search Architecture for Grid Software Components.
this paper, we are going to discuss the challenges we have to face in designing a search service for locating software components on the Grid. Indeed, the specifications of our search engine rely heavily on the concept of Ecosystem of Components. Basically it consists in an extension of the marketplace concept to the Grid. The ecosystem, in fact, provides applications a virtual environment where they can live (i.e. are executed), and find other components to which they can refer (i.e. cooperate) to build larger (and more complex) systems
Design of a Parallel and Distributed Web Search Engine
This paper describes the architecture of MOSE (My Own Search Engine), a scalable parallel and distributed engine for searching the web. MOSE was specifically designed to efficiently exploit affordable parallel architectures, such as clusters of  workstations. Its modular and scalable architecture can be easily adjusted to fulfill the bandwidth requirements of the application at hand. Both task-parallel  and data-parallel approaches are exploited within MOSE in order to increase  the throughput and efficiently use communication, storing and computational resources. We used
kDCI a Multi%2DStrategy Algorithm for Mining Frequent Sets.
09:00 Welcome, introduction, results of the global experiments 10:30 Coffee break 11:00 Panel Session I (session chair: tba) &#034;A fast APRIORI implementation&#034;, by Ferenc Bodon &#034;Efficient Implementations of Apriori and Eclat&#034;, by Christian Borgelt
A Hybrid Strategy for Caching Web Search Engine Results.
This work discusses the design and implementation of an efficient caching system aimed to exploit the locality present in the queries submitted to a Web Search Engine (WSE). We enhance previous proposals in several directions. First we propose the adoption of a hybrid strategy for caching, and then we experimentally demonstrate the superiority of our hybrid strategy. Further we show how to take advantage of the spatial locality present in WSE query logs by exploiting a sort of adaptive prefetching strategy.
Scheduling High Performance Data Mining Tasks on a Data Grid Environment.
Increasingly the datasets used for data mining are becoming huge and physically  distributed. Since the distributed knowledge discovery process is both data and computational  intensive, the Grid is a natural platform for deploying a high performance data mining  service. The focus of this paper is on the core services of such a Grid infrastructure. In particular  we concentrate our attention on the design and implementation of specialized Resource  Allocation and Execution Management services aware of data source locations and resource  needs of data mining tasks. Allocation and scheduling decisions are taken on the basis of  performance cost metrics and models that exploit knowledge about previous executions, and  use sampling to acquire estimate about execution behavior.
Adaptive and Resource%2DAware Mining of Frequent Sets.
The performance of an algorithm that mines frequent sets from transactional databases may severely  depend on the specific features of the data being analyzed. Moreover, some architectural characteristics  of the computational platform used -- e.g. the available main memory -- can dramatically change the  runtime behaviors of the algorithm. In this paper we present DCI (Direct Count &amp; Intersect), an e#cient  data mining algorithm for discovering frequent sets from large databases, which e#ectively addresses the  issues mentioned above. DCI adopts a classical level-wise approach based on candidate generation to  extract frequent sets, but uses a hybrid method to determine candidate supports. The most innovative  contribution of DCI relies on the multiple heuristics strategies employed, which permits DCI to adapt its  behavior not only to the features of the specific computing platform, but also to the features of the dataset  being mined, so that it results e#ective in mining both short and long patterns from sparse and dense  datasets. The large amount of tests conducted permit us to state that DCI sensibly outperforms state-ofthe  -art algorithms for both synthetic and real-world datasets. Finally we also discuss the parallelization  strategies adopted in the design of ParDCI, a distributed and multi-threaded implementation of DCI.
An Efficient Parallel and Distributed Algorithm for Counting Frequent Sets.
Due to the huge increase in the number and dimension of  available databases, e#cient solutions for counting frequent sets are nowadays  very important within the Data Mining community. Several sequential  and parallel algorithms were proposed, which in many cases exhibit  excellent scalability. In this paper we present ParDCI, a distributed and  multithreaded algorithm for counting the occurrences of frequent sets  within transactional databases. ParDCI is a parallel version of DCI (Direct  Count &amp; Intersect), a multi-strategy algorithm which is able to adapt its  behavior not only to the features of the specific computing platform (e.g.
